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Abstract 

We consider an isolated system in an arbitrary state and provide a general formulation using first principles for an additive 
and non-negative statistical quantity So{t) that is shown to reproduce the equilibrium thermodynamic entropy of the isolated 
system. We further show that So{t) represents the nonequilibrium thermodynamic entropy So{t) when the latter is a state 
function of nonequilibrium state variables; see text. We consider an isolated 1-d ideal gas and determine its non-equilibrium 
statistical entropy <So(i) as a function of the box size as the gas expands freely isoenergetically, and compare it with the 
equilibrium thermodynamic entropy Socq- We find that So{t) < Soeq in accordance with the second law, as expected. To 
understand how So{t) is different from thermodynamic entropy of classical continuum models that is known to become negative 
under certain conditions, we calculate So for a 1-d lattice model and discover that it can be related to the thermodynamic 
entropy of the continuum 1-d Tonks gas by taking the lattice spacing S go to zero. However, ASo 7^ ASo since S is state- 
dependent. We discuss the semi-classical approximation of our entropy and show that the standard quantity Sf{t) in the 
Boltzmann's H-theorem, see Eq. ((3|, does not directly correspond to the statistical entropy. 
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I. INTRODUCTION 

Although the concept of entropy plays important roles 
in diverse fields ranging from classical thermodynamics 
of ClausiuSfi"— quantum mechanics and uncertainty,—^— 
black holesj^ coding and computation^iSiiii to informa- 
tion technologyp^iii^ it does not seem to have a standard 
definition. Here, we are interested in its application to 
nonequilibrium statistical thermodynamics. In classical 
thermodynamics, it is defined as a thermodynamic quan- 
tity with no association of any notion of a microstate 
and its probability, while modern approach to statisti- 
cal thermodynamics, primarily due to Boltzmann and 
Gibbs, requires a probabilistic approach in terms of mi- 
crostates. In this work, we are primarily interested in 
an isolated system Eg. Quantities for the isolated sys- 
tem will carry a suffix 0; quantities without the suffix 
will refer to any body, which need not be isolated, such 
as a part E of Eq; see Fig. [T] The microstates for Eq 
are determined by the set Xq of extensive observables 
(the energy Eq, the volume Vb, the number of particles 
Nq , etc.) specifying the isolated system. While tem- 
poral evolution is not our primary interest in this work, 
we still need to remember the importance of temporal 
evolution in thermodynamics. We will say that two mi- 
crostates belonging to the microstate subspace ro(Xo) 
are "connected" if one evolves from the other after some 
time tq < 00. Before this time, they will be treated as 
"disconnected." Let romax denote the maximum tq over 
all pairs of microstates. The space ro(Xo) is simply con- 
nected for all times longer than Tomax in that each mi- 
crostate can evolve into another microstate S ro(Xo) in 
due time. For t < romax, the space ro(Xo) will consist of 
disjoint components, an issue that neither Boltzmann nor 
Gibbs has considered to the best of our knowledge. But 
the issue, which we consider later in Sec. IIIIBl becomes 
important in considering nonequilibrium states. 

Boltzmann assumes equal probability of various mi- 
crostates in the simply connected set ro(Xo). Thus, 
Tomax can be identified with the equilibration time 




FIG. 1: Schematic representation of S, E and Eq. We assume 
that S and E are homogeneous and in internal equilibrium, 
but not in equilibrium with each other. The internal fields 
T{t),P{t),--- fof E and To,Po,--- of S are not the same 
unless they are in equilibrium with each other. There will be 
viscous dissipation in E when not in equilibrium with E. 



Tocq(Xo) for Eq. Under the equiprobable assumption, 
Boltzmann identifies the entropy in terms of the num- 
ber of microstates^^i^ Wq (Xq) in Fq (Xq): 



5oB(Xo)=lnIFo(Xo); 



(1) 



we will set the Boltzmann constant to be unity through- 
out the work so that the entropy will always be a pure 
number. The idea behind the above formula implic- 
itly appears for the first time in a paper— by Boltz- 
mann, and then appears more or less in the above form 
later in his lectures^^ where he introduces the combi- 
natorial approach for the first time to statistical me- 
chanics. The formula itself does not appear but is im- 
plied when he takes the logarithm of the number of 
combinations.— lii (There is another formulation for en- 
tropy given by Boltzmann ;^^'^^ which is also known as 
the Boltzmann entropy,— that we will discuss later and 
that has a restricted vafidity; see Eq. (|40| .) Gibbs, also 
using the probabilistic approach, gives the following for- 
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mula for the entropy in a canonical ensemble)^ 

4'^^- EpL^^inpL^^ Epi^^ = i (2) 
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where pa is the canonical ensemble probability of the 
ath microstate of the system, and the sum is over all mi- 
crostates corresponding to all possible energies E (with 
other elements in X held fixed); their set is denoted by 
Fe in the above sum. The Gibbsian approach assumes 
an ensemble at a given instant, while the Boltzmann ap- 
proach considers the evolution of a particular system in 
time; see for example a recent review.^^ In equilibrium, 
both entropy expressions yield the same result. In quan- 
tum mechanics, this entropy is given by the von Neumann 
entropy formulation^'^ in terms of the density matrix p: 



S. 



-Tr{p\np). 



The entropy formulation in the information theor y ^^i^'^ 
has a form that appears to be similar in form to the above 
Gibbs entropy even though the temperature has no sig- 
nificance in the information theory. There is also another 
statistical formulation of entropy, heavily used in the lit- 
erature, in terms of the phase space distribution func- 
tion f{x,t), which follows from Boltzmann's celebrated 
H-theorem: 



Sf{t) = ~Jfix,t)\nf{x,t)dx; 



(3) 



here, x denotes a point in the phase space. This quantity 
is not only not dimensionless but, as we will show later, 
is not the correct formulation in general; see Eq. p9ap . 
The classical thermodynamics entropy Sq is oblivious 
to the microstates and their probabilities and deals with 
Eo,Vo,No, etc. as the observables of the system. In equi- 
librium, the entropy Sq is a state function, and can be ex- 
pressed as a function of the observables. This functional 
dependence results in the Gibbs fundamental relation 



dSo = Ep {dSo/dXop) dXop 



(4) 



in terms of the observables {Xop}. For a lattice model, 
Sq is non-negative in accordance with the Boltzmann def- 
inition of Sob (Xq), but is known to become negative for 
a continuum model such as for an ideal gas. The latter 
observation implies that such continuum models are not 
realistic as they violate Nernst's postulate (the third law) 
and require quantum mechanics to ensure non-negativity 
of the entropy.- Even the change AS'o, the heat capac- 
ity, etc. do not satisfy thermodynamic consequences of 
Nernst's postulate. 

By invoking Nernst's postulate (the equilibrium en- 
tropy vanishes at absolute zero), one can determine the 
equilibrium entropy everywhere uniquely. The consen- 
sus is that in equilibrium, the thermodynamic entropy is 
not different from the above statistical entropies due to 
Boltzmann and Gibbs. However, there is at present no 
consensus when the system is out of equilibrium. There 



is also some doubt whether the nonequilibrium thermo- 
dynamic entropy has any meaning in classical thermody- 
namics. We will follow Clausius and take the view here 
that the thermodynamic entropy is a well-defined notion 
even for an irreversible process going on in a body for 
which Clausiusi (p. 214) writes TdS > d^Q in terms of 
the exchange heat dcQ with the medium. The question 
that arises is whether the two statistical definitions can 
be applied to a body out of equilibrium. We find the 
answer to be afRrmative. The next question that arises 
is the following: Do they always give the same results? 
We will show that under certain conditions, they give the 
same results. This is important as the literature is not 
very clear on this issue<^"— 

For an isolated system, which we mostly consider here, 
we are not concerned with any thermostat or external 
working source. As a consequence, observables i?o,Vo,iVo, 
etc. in Xq must remain constant even if the system is 
out of equilibrium. While this will simplify our discus- 
sion to some extent, it will also create a problem as we 
will discuss later. We discuss the concept of a nonequi- 
librium state, state variables and state functions in the 
next section. We introduce the statistical entropy for- 
mulation in Sec. Ill and show its equivalence with ther- 
modynamic nonequilibrium entropy when the latter is a 
state function. In Sec. IV, we carry out an explicit quan- 
tum calculation of the nonequilibrium statistical entropy. 
In Sec. V, we consider a 1-d lattice model appropriate 
for Tonks gas in continuum so that the statistical lat- 
tice entropy can be calculated rigorously. We take the 
continuum limit and compare the resulting entropy with 
the continuum entropy of the Tonks gas and obtain an 
interesting result. We discuss the semi-classical approx- 
imation of the statistical entropy in Sec. VI and show 
that the formulation in Eq. ([3]) does not determine the 
entropy. A brief summary and discussion is presented in 
the final section. 



II. THE SECOND LAW AND A NONEQUILIB- 
RIUM STATE 

A. Second Law 

The second law states that the irreversible (denoted by 
a sufRx i) entropy generated in any infinitesimal physical 
process going on in a body satisfies the inequality 



diS > 0; 



(5) 



the equality occurs for a reversible process. For an iso- 
lated system, there is no exchange (denoted by a suf- 
fix e) entropy change d^So with the medium so that 
dSo = dcSo + diSo in any arbitrary process, in an isolated 
system satisfies 



dSo = diSo > 0. 



(6) 



The law refers to the thermodynamic entropy. It is not 
a state function in the conventional sense (S'o(Xo)) if Eq 



is not in equilibrium simply because as a state function 
S'o(Xo) must remain constant for constant Xq; see below 
also. 

As the thermodynamic entropy is not measurable ex- 
cept when the process is reversible, the second law re- 
mains useless as a computational tool. In particular, it 
says nothing about the rate at which the irreversible en- 
tropy increases. Therefore, it is useful to obtain a com- 
putational formulation of the entropy, the statistical en- 
tropy. This will be done in the next section. The onus 
is on us to demonstrate that the statistical entropy also 
satisfies this law if it is to represent the thermodynamic 
entropy. This by itself does not prove that the two are 
the same. It is not been possible to show that the statisti- 
cal entropy is identical to the thermodynamic entropy in 
general. Here, we show their equivalence only when the 
nonequilibrium thermodynamic entropy is a state func- 
tion of nonequilibrium state variables to be introduced 
below. 



B. Concept of a Nonequilibrium State 

For an isolated body in equilibrium, the entropy can be 
expressed as a function of its observables (variables that 
can be controlled by an observer), as is easily seen form 
the Gibbs fundamental relation in Eq. (j4]). The ther- 
modynamic state, also known as the macrostate Mo, of 
a body in equilibrium remains the same unless it is dis- 
turbed. Therefore, we can identify the equilibrium state 
A^Ooq of the body by its observables. Accordingly, its 
equilibrium entropy Sq(X.q) can be expressed as a func- 
tion of its observables Xq. This is expressed by saying 
that the equilibrium entropy is a state function and Xq 
is the set of state variables. 

The above conclusion is most certainly not valid for a 
body out of equilibrium, which we take to be isolated. If 
the body is not in equilibrium, its (macro) state Adoit) 
will continuously change, which is reflected in its en- 
tropy change (increase) in time; this requires expressing 
its entropy as S'o(Xo, t) with an explicit time-dependence, 
since Xq = constant for an isolated body. The change 
in the entropy and the macrostate must come from the 
variations of additional variables, distinct from the ob- 
servables, that keep changing with time until the body 
comes to equilibrium as explained elsewhere j^i^ These 
variables cannot be controlled by the observer. Once the 
body has come to equilibrium, the entropy has no ex- 
plicit time-dependence and becomes a state function. In 
this state, the entropy has its maximum possible value 
for given Xq. In other words, when the entropy becomes 
a state function, it achieves the maximum possible value 
for the given set of state variables, here given by Xq. This 
conclusion about the entropy will play an important role 
below. 

We assume that there is a set ^g of additional vari- 
ables, known as the internal variables (sometimes also 
called hidden variables). We will refer to the variables in 



Xq and ^q as {nonequilibrium) state variables (see below 
for justification) and denote them collectively as Zq in 
the following. From Theorem 4 presented elsewhere?^ it 
follows that with a proper choice of the number of in- 
ternal variables, the entropy can be written as S'o(Zo(i)) 
with no explicit t-dependence. The situation is now al- 
most identical to that of an isolated body in equilibrium: 
The entropy is a function of Zq (t) with no explicit time- 
dependence. This allows us to identify Zo(i) as the set 
of nonequilibrium state variables. Thus, A4o(t) can be 
specified by Zo(i) so that the entropy becomes a state 
function. This allows us to extend Eq. @ to 



^5*0 = Ep {dSo/dZ„p) dZ, 



Op 



(7) 



in which the partial derivatives are related to the fields 
of the system: 



(dSo/dEo) = l/To, (dSo/dVo) = Pq/To, • 



(8) 



these fields will change in time unless the system has 
reached equilibrium. 

As Zo(i) changes in time, Aio{t) changes, but at each 
instant the (nonequilibrium) entropy as a state func- 
tion, has a maximum possible value for given Zo(i) even 
though M.o{t) ^ A^oeq- In our previous work,^^"^ we 
have identified this particular state as an internal equi- 
librium state, but its physical significance as presented 
above was not discussed. For a state that is not in in- 
ternal equilibrium, the entropy must retain an explicit 
time-dependence. In this case, the derivatives in Eq. ([5]) 
cannot be identified as state variables like, temperature, 
pressure, etc. 

It may appear to a reader that the concept of entropy 
being a state function is very restrictive. This is not the 
case as this concept, although not recognized by several 
workers, is implicit in the literature where the relation- 
ship of the thermodynamic entropy with state variables 
is investigated. To appreciate this, we observe that the 
entropy of a body in internal equilibrium^li^ is given by 
the Boltzmann formula 



SiZit))=\nW{Z{t)), 



(9) 



in terms of the number of microstates corresponding 
to Z(i). In classical nonequilibrium thermodynamics^ 
the entropy is always taken to be a state function. In 
the Edwards approach^^ for granular materials, all mi- 
crostates are equally probable as is required for the above 
Boltzmann formula. Bouchbinder and Langer— assume 
that the nonequilibrium entropy is given by Eq. ^. 
Lebowitz^'^ also takes the above formulation for his def- 
inition of the nonequilibrium entropy. As a matter of 
fact, we are not aware of any work dealing with entropy 
computation that does not assume the nonequilibrium 
entropy to be a state function. This does not, of course, 
mean that all states of a system are internal equilibrium 
states. For states that are not in internal equilibrium, 
the entropy is not a state function so that it will have 
an explicit time dependence. But, as shown elsewhere^ 



this can be avoided by enlarging the space of internal 
variables. The choice of how many internal variables are 
needed will depend on experimental time scales and can- 
not be answered in generality at present. We hope to 
come back to this issue in a future publication. 

For a general body that is not isolated, the concept of 
its internal equilibrium state plays a very important role 
in that the body can come back to this state several times 
in a nonequilibrium process. In a cyclic nonequilibrium 
process, such a state can repeat itself in time after some 
cycle time t,. so that all state variables and functions 
including the entropy repeat themselves: 

Z(i + Te) = Z(i), M{t + Tc) = M{t), S(t + Te) = S[t). 

This ensures Ac5 = S{t + t^) — S{t) = in a cyclic 
process. All that is required for the cyclic process to 
occur is that the body must start and end in the same 
internal equilibrium state; however, during the remainder 
of the cycle, the body need not be in internal equilibrium. 



III. GENERAL FORMULATION OF THE STA- 
TISTICAL ENTROPY 

We provide a very general formulation of the statis- 
tical entropy, which will also demonstrate that the en- 
tropy is a statistical average. We consider a macrostate 
M.o{t) = A^o(Zo(i)) of Eq at a given instant t. In the fol- 
lowing, we suppress t unless necessary. The macrostate 
A4o(Zo) refers to the set of microstates mo — {tooq} 
and their probabilities p = {pa}- For the computa- 
tion of combinatorics, the probabilities are handled in 
the following abstract way. We consider a large num- 
ber A/o= CoW^o(Zo) of independent replicas or samples 
of Eqj with Co some large constant integer and W^o(Zo) 
the number of distinct microstates moa- The samples 
should be thought of as identically prepared experimental 
samples.— Let ro(Zo) denote the sample space spanned 
by moa- 



A. Simply 



Connected 



Sample 



Space 



1. An Isolated System 

As ro(Zo) C ro(Xo), ro(Zo) is simply connected if 
ro(Xo) is, which we assume in this section. Let A/oa(i) 
denote the number of TTioa-samples (samples in the moa- 
microstate) so that 



Q<Pc.{t)=Moc{t)/Mo<l; 



Wo(Zo) 

E A/'oc.(t)=A/'o. (10) 



The above sample space is a generalization of the en- 
semble introduced by Gibbs, except that the latter is 
restricted to an equilibrium system, whereas our sample 
space refers to the system in any arbitrary state so that 



Pa may be time-dependent. In the semi-classical approx- 
imation, see Sec. I VII one can similarly take the sample 
space to represent the classical phase space of Boltzmann. 
The {sample or ensemble) average of some quantity Qo 
over these samples is given by 



Qc 



Wo(Zo) Wo(Zo) 



a=l 



(11) 



where Qoa is the value of Qo in 'tioq ■ 

The samples are, by definition, independent of each 
other so that there are no correlations among them. Be- 
cause of this, we can treat the samples to be the outcomes 
of some random variable, the macrostate A^o(i)- This 
independence property of the outcomes is crucial in the 
following, and does not imply that they are equiprobable. 
The number of ways Wq to arrange the A/o samples into 
Wo(Zo) distinct microstates is 



Wo=AAo!/nA/'oa(0! 



(12) 



Taking its natural log to obtain an additive quantity per 
sample 



5o = lnWo/A/'o, 



(13) 



and using Stirling's approximation, we see easily that 
(So, which we hope to identify later with the entropy 
S'o(Zo(t)) of the isolated system, can be written as the 
average of the negative of 

r;(i) = Inp(i), 

what Gibbs'^ calls the index of probability: 



So{Zo{t),t) = ~r,it) 



Wo(Zo 

- E 



p„(t)lnp„(i), (14) 



where we have also shown an explicit time-dependence 
for the reason that will become clear below. The above 
derivation is based on fundamental principles and does 
not require the system to be in equilibrium; therefore, it 
is always applicable. To the best of our knowledge, even 
though such an expression has been extensively used in 
the literature, it has been used without any derivation; 
one simply appeals to this form by invoking it as the 
information entropy; however, see Sec. I Villi 

Because of its similarity in form with Sq in Eq. ^ , 
we will refer to iSo(Zo(i),i) as the Gibbs statistical en- 
tropy from now on. As the nonequilibrium thermody- 
namic entropy for a process in which the system is al- 
ways in internal equilibrium can be determined by inte- 
grating the Gibbs fundamental relation in Eq. ([7|), we 
can compare it with the statistical entropy introduced 
above. However, such an integration is not possible for a 
process involving states that are arbitrary (not in inter- 
nal equilibrium). Therefore, there is no meaning to com- 
pare 5o(Zo(i),i) with the corresponding thermodynamic 
entropy whose value cannot be determined. To identify 
iSo(Zo(i)) with the nonequilibrium thermodynamic en- 
tropy requires the following additional steps: 



(1) It is necessary to establish that 5o(i) satisfies Eq. 
©■ 

(2) For an equihbrium canonical system, it is necessary 
to establish that S(t) is identical to the equilibrium 
thermodynamic entropy given by Sq ^ 

(3) It is necessary to show that So{t) is identical to 
the nonequilibrium thermodynamic entropy of the 
system that is out of equilibrium but whose entropy 
is a state function. 

There are several proofs available in the 
liter ature^ii^ii^i^ for (1). Therefore, we will not 
be concerned with (1) anymore. We will prove (2) and 
(3) in Sect. IIIIA2I 

The maximum possible value of Sq (t) for given Zq (t) 
occurs when moa are equally probable: 



Pa{t) ->Po 



l/l^o(Zo), Vm„ e ro(Zo). 



In this case, the explicit time dependence in So(t) will 
disappear and we have 

5o,max(Zo(i),t) =5o(Zo(i)) = lnWo(Zo(i)), (15) 

which is identical in form to the Boltzmann (thermody- 
namic) entropy in Eq. ([T]) for an isolated body in equi- 
librium, except that the current formulation has been 
extended to an isolated body out of equilibrium; see also 
Eq. 0. The only requirement is that all microstates in 
mo = mo(Zo(i)) are equally probable. The statistical 
entropy in this case becomes a state function. 

Applying the above formulation to a macrostate char- 
acterized by a given Xq and consisting of microstates 
{m„} , forming the set mo = mo(Xo), with probabilities 
{Pa{t)}, we find that 

Wo(Xo) Wo(Xo) 

5o(Xo,t) = - E p„(t)lnp„(i), Y: p^{t) = l, 

(16) 
is the entropy of this macrostate, where WqCKq) is the 
number of distinct microstates m^ . It should be obvious 
that 

Wo(Xo) ^ Ei,it)WoiZo{t)). 

Again, under the equiprobable assumption 

Pa{t) -^ Pa,cq = l/Wo(Xo), VrUa E ro(Xo), 

ro(Xo) denoting the space spanned by microstates {ma}, 
the above entropy takes its maximum possible value 



5o,max(Xo,i) = 5o(Xo) = In Wo(Xo), 



(17) 



which is identical in value to the Boltzmann (thermody- 
namic) entropy in Eq. ([T]) for an isolated body in equi- 
librium. The maximum value occurs at t = Tooq. It is 
evident that 



The anticipated identification of nonequilibrium ther- 
modynamic entropy with Sq under some restrictions al- 
lows us to identify Sq as the statistical entropy formu- 
lation of the thermodynamic entropy. From now on, we 
will refer to the general entropy in Eq. (J14p in terms 
of microstate probabilities as the time- dependent Gibbs 
formulation of the entropy or simply the Gibbs entropy, 
and will not make any distinction between the statistical 
and thermodynamic entropies. Accordingly, we will now 
use the regular symbol 5*0 for iSo throughout the work, 
unless clarity is needed. 

We will refer to S'o(Zo(t)) in terms of microstate num- 
ber T4^o(Zo(0) i^ Eq. p5l) as the time- dependent Boltz- 
mann formulation of the entropy or simply the Boltz- 
mann entropy,^'^ whereas 5*0 (Xo) in Eq. (|17p represents 
the equilibrium (Boltzmann) entropy. It is evident that 
the Gibbs formulation in Eqs. ([H]) and (ITBl) supersedes 
the Boltzmann formulation in Egs. lfTS]) and p7|) . respec- 
tively, as the former contains the latter as a special limit. 
However, it should be also noted that there are com- 
peting views on which entropy is more general<22i2^ We 
believe that the above derivation, being general, makes 
the Gibbs formulation more fundamental. The continu- 
ity of S'o(Zo,t) follows directly from the continuity of 
Pa{t)- The existence of the statistical entropy 5'o(Zo,t) 
follows from the observation that it is bounded above by 
In Wo(Zo) and bounded below by 0, see Eq. P^ . 

It should be stressed that W is not the number of 
microstates of the N replicas; the latter is given by 
[Wo(Zo)]^. Thus, the entropy in Eq. ^ should not 
be confused with the Boltzmann entropy, which would 
be given by A/'ln Wo(Zo). It should be mentioned at this 
point that Boltzmann uses the combinatorial argument 
to obtain the entropy of a gas, see Eq. ([M)) . resulting 
in an expression similar to that of the Gibbs entropy in 
Eq. ^ except that the probabilities appearing in his 
formulation represents the probability of various discrete 
states of a particle, and should not be confused with the 
microstate probabilities used here; see Sec. I VIII The ap- 
proach of Boltzmann is limited to that of an ideal gas 
only and is not general as it neglects the correlations 
present due to the interactions between particlesJ^i^ 
On the other hand, our approach is valid for any sys- 
tem with any arbitrary interactions between particles as 
all microstates in the collection are independent. 



2. System in a Medium and Quasi-independence 

Using the above formulation of So{Zo{t),t), we have 
determinec^ the statistical formulation of the entropy 
for a system S, which is a small but macroscopically large 
part of Eo; see Fig. [1] It is assumed that the system 
and the medium are quasi-independent so that So{t) can 
be expressed as a sum of entropies S{t) and S{t) of the 
system and the medium, respectively: 



So[Zoit),t] <5o[Zo(t)]<5o(Xo). 



(18) 



So{t) = S{t) + Sit). 



(19) 



The two statistical entropies are given by an identical 
formulation 



k 



S{t) 



-j:Pk{t)\npk{t), 

k 

(20) 
respectively. Here, nik with probability pk denotes a mi- 
crostate of E and fhk with probability pk that of the 
medium. In the derivation,-^ we have neither assumed 
the medium nor the system to be in internal equilibrium; 
only quasi-independence is assumed. The above formu- 
lation of statistical entropies will not remain valid if the 
two are not quasi-independent. The same will also be 
true of the thermodynamic entropies. 

For the system to be in internal equilibrium, its sta- 
tistical entropy S{t) must be maximized under the con- 
straints imposed by the medium. The constraints are on 
the average values of the state variables: 



Z(0 ^^'^kPk, 



Moreover, as the nonequilibrium entropy in internal equi- 
librium is a state function, it can in principle be measured 
or calculated by integrating Eq. ([^ . Therefore, its value 
can be compared with the statistical entropy. The above 
identification in Eq. (|23l) then proves (3). 

If the thermodynamic entropy is not a state function, 
it cannot be measured or computed. Thus, while the sta- 
tistical entropy can be computed in principle in all cases, 
as shown below explicitly, there is no way to compare 
its value with the thermodynamic entropy in all cases. 
Thus, no comment can be made about their relationship 
in general. We merely conjecture that as the two en- 
tropies are the same when the thermodynamic entropy 
is a state function, it is no different from its statistical 
analog even when it is not a state function. 



B. Disjoint Sample Space (Component Confine- 
ment) 



where Z^ is the value of Z in mk- The condition for 
internal equilibrium is obtained by varying pk without 
changing the microstates, i.e. Z^. Using the Lagrange 
multiplier technique, it is easy to see that the condition 
for this in terms of the Lagrange multipliers whose defi- 
nitions are obvious is 



Inpk = Ao -I- \-Zk 



(21) 



the Lagrange multipliers are the same for all microstates 
and the scalar product is over the elements in the set Zk ■ 
It now follows that 



S^Xa + X-Z{t), 



(22) 



using the same scalar product as above. It is now easy 
to identify the Lagrange multipliers by observing that 



dS = X-dZ. 



(23) 



Comparing this relation with the Gibbs fundamental re- 
lation for the system, which follows from Eq. (O when 
applied to the system, we find 

Ap = (dS/dZp) . 

Accepting this identification now allows us to conclude 
that the statistical entropy S{t) in Eq. ([^0)1 is no different 
than the nonequilibrium thermodynamic entropy of the 
same system in internal equilibrium but in a medium. A 
special case of such a system is the (equilibrium) canon- 
ical ensemble of Gibbs. This proves (2) mentioned in 
Sect, nil A II In equilibrium, the Lagrange multipliers 
associated with the internal variables vanish and Eq. ([25)1 
reduce to 



dS = A-dX. 



(24) 



The significance of Aq is quite obvious. In internal equi- 
librium, it is given by 

Ao = 5* - (dS/dZ) ■ Z(t). 



The consideration of dynamics resulting in the simple 
connectivity of the sample (or phase) space has played a 
pivotal role in developing the kinetic theory of gases^^i^ 
where the interest is at high temperatures i^ '^^i'^"'^"'^ As 
dynamics is very fast here, it is well known that the 
ensemble entropy agrees with its temporal formulation. 
However, at low temperatures, where dynamics becomes 
sluggish as in a glass,'^^'^- the system can be confined into 
disjoint components. 

Sample (or phase) space confinement at a phase transi- 
tion such as a liquid-gas transition is well known in equi- 
librium statistical mechanics j^^i^i^ It also occurs when 
the system undergoes symmetry breaking such as dur- 
ing magnetic transitions, crystallizations, etc. But con- 
finement can also occur under nonequilibrium conditions, 
when the observational time scale Tobs becomes shorter 
than the equilibration time Teqr^^ such as for glasses, 
whose behavior and properties have been extensively 
studied. 

The issue has been recently considered by uSfiS where 
only energy as an observable was considered. The discus- 
sion is easily extended to the present case when confine- 
ment occurs for whatever reasons into one of the thermo- 
dynamically significant number of disjoint components 
LoA, A = 1,2,3--- ,C, each component corresponding to 
the same set Zq or Z (we suppress the dependence for 
simplicity), depending on whether the body is isolated 
or not. Such a situation arises, for example, in Ising 
magnets at the ferromagnetic transition., where the sys- 
tem is either confined to ro+ with positive magnetization 
or To- with negative magnetization. Even a weak exter- 
nal magnetic field \H\ -^ 0, that we can control as an 
observer, will allow the system to make a choice between 
the two parts of Fq. It just happens that in this case 
C = 2 and is thermodynamically insignificant. 

The situation with glasses or other amorphous mate- 
rials is very different. '^^ In the first place, Fq is a union 
of thermodynamically significant number C ^ e'^'^ ,c > 0, 



disjoint components. In the second place, there is no 
analog of a symmetry breaking field. Therefore, there is 
no way to prepare a sample in a given component Fqa- 
Thus, the samples will be found in all different compo- 
nents. Taking into consideration disjointncss of the com- 
ponents generalizes the number of configurations in Eq. 
(|T2|1 to 

Wo=Aro!/nM)a.(i)!, 

where TVoqa denotes the number of sample in the mi- 
crostate rria^ in the A-th component. In terms of p^^ = 
A/'oq;^ (t)/A/o, this combination immediately leads to 

Soit) = InWo/AAo = -EAEa.Pa. WlnPa.(i), (25) 

for the statistical entropy of the system and has already 
been used earlier— by us; see Sec. 4.3.3 there. From what 
has been said above, this statistical entropy is also the 
thermodynamic entropy of a nonequilibrium state under 
component confinement for which the entropy is a state 
function of Zq. Therefore, as before, we take So to be the 
general expression of the nonequilibrium thermodynamic 
entropy and use 5*0 in place of Sq . 
Introducing 

Pa(0 = Ea;,P"A(i)> 

it is easy to see^^ that 

So{t)^ExPxit)Sox{t) + Sc{t). 

Here, the entropy of the component Fqa in terms of the 
reduced microstate probability pa^ =pa^/p\ is 



So\{t) = -Ea^P^AWlnPaAW 



(26) 



so that the first contribution is its average over all com- 
ponents. The second term is given by 



•^cW^-EAPA^lnpAW, 



(27) 



and represents the component entropy. It is this entropy 
that is related to the residual entropy'^'' in disordered 
systems. The same calculation for a system in a medium 
will result in an identical formulation for the entropy as 
in Eq. ([25)) except that the sum is over components and 
microstates of the system. 



IV. 1-DIMENSIONAL IDEAL GAS: A MODEL 
ENTROPY CALCULATION 

We consider a gas of non-interacting identical struc- 
tureless particles with no spin, each of mass tti, confined 
to a 1-dimensional box of initial size L = Ljn with impen- 
etrable walls (infinite potential well) . Initially, the gas is 
in thermodynamic equilibrium with a medium at fixed 
temperature Toin and pressure Poin- The gas is then iso- 
lated by disconnecting it from the medium. In time, the 



isolated gas expands, may be in a nonequilibrium fash- 
ion. We wish to calculate its entropy as a function of the 
box size L{t). 

As there are no interactions between the particles, the 
wavefunction ^ for the gas is a product of individual par- 
ticle wavefunctions ip. Thus, we can focus on a single par- 
ticle to study the nonequilibrium behavior of the gas.— i^ 
The simple model of a particle in a box has been exten- 
sively studied in the literature but with a very differ- 
ent emphasis.—"— The particle only has non-degenerate 
eigenstates whose energies are determined by L = aLin, 
a > 1, and a quantum number k. We use the en- 
ergy scale ei — 7r^fi,^/2mL?j to measure the energy of 
the eigenstate so that 

£fc(i) = fcVa^; (28) 

the corresponding eigenfunctions are given by 



ipkix) = ^/2jLsm{kTrx/L), fc = 1, 2, 3, 



(29) 



The pressure generated by the eigenstate on the walls is 
given by^ 



Pk{L) = -dek/dL = 2ek{L)/L. 



(30) 



In terms of the eigenstate probability Pk{t), the average 
energy and pressure are given by 

£(i,i)=EfcPfc(i)£fc(i), (31a) 

Pit, L) = EkPkit)PkiL) = 2e(i, L)/L. (31b) 

The entropy follows from Eq. (|14p and is given for the 
single particle case by 

s{t,L) = -J2^Pk{t)\npk{t). 

The time dependence in e{t) or P{t) is due to the time 
dependence iiapk and ek{L). Even for an isolated system, 
for which e remains constant, pk cannot remain constant 
when the gas is not in equilibrium if L is held fixed after 
expansion. This follows directly from the second lawi^ 
and creates a conceptual problem because the eigenstates 
are mutually orthogonal and there can be no transitions 
among them to allow for a change in pk ■ 



A. Chemical Reaction Approach 

A way to change pk in an isolated system is to re- 
quire the presence of some stochastic interactions, whose 
presence allows for transitions among eigenstates.^^ As 
these transitions are happening within the system, we 
can treat them as "chemical reactions" between differ- 
ent eigenstates^Zi^iiS by treating each eigenstate k as 
a chemical species. During the transition, these species 
undergoes chemical reactions to allow for the changes in 
their probabilities. 

We follow this analogy further and extend the tradi- 
tional approac h^^'--'"- to the present case. For the sake of 



simplicity, our discussion will be limited to the ideal gas 
in a box; the extension to any general system is trivial. 
Therefore, we will use microstates instead of eigenstates 
in the following to keep the discussion general. Let there 
be Nk{t) particles in the fcth microstate at some instant 
t so that 

at all times, and Pk{t) — Ni^{t)/N . We will consider the 
general case that also includes the case in which final mi- 
crostates refer to a box size L' different from its initial 
value L. Let us use Ak to denote the reactants (initial 
microstates) and A'^ to denote the products (final mi- 
crostates). For the sake of simplicity of argument, we 
will assume that transitions between microstates is de- 
scribed by a single chemical reaction, which is expressed 
in stoichiometry form as 



Efcflfc^fc — ^ Efe4^fe- 



(32) 



Let Nk and N'j, denote the population of A}, and AJ., 



respectively, so that N = Efc-^fc 



T.kK- Accord- 



ingly, Pk{t) = Nk{t)/N for the reactant and pk{t + dt) — 
Nl^{t)/N for the product. The single reaction is described 
by a single extent of reaction ^ and we have 

d^{t) = -dNk{t)/ak{t) = dNl,{t)/a'^,,{t) for all fc,fc'. 

It is easy to see that the coefficients satisfy an important 
relation 

Efcafc(0 = EfeOfeW: 

which reflects the fact that the change \dN\ in the re- 
actant microstates is the same as in the product mi- 
crostates. The affinity in terms of the chemical potentials 
is given by 

A{t) = E«fcWMA,(t) -E4Wa^a^W, 

and will vanish only in "equilibrium," i.e. only when pk 
s attain their equilibrium values. Otherwise, A{t) will 
remain non-zero. It acts as the thermodynamic force 
in driving the chemical reactioni^i^Si^ But we must 
wait long enough for the reaction to come to comple- 
tion, which happens when A{i) and d^/dt both vanish. 
The extent of reaction ^ is an example of an internal vari- 
able. For the ideal gas under consideration, there does 
not seem to be any other internal variable as particles 
have no internal structures. In the following, we will as- 
sume only one internal variable £,{t). 



B. Free (Sudden) Expansion of the Box 

The box expands as a function of time, which need 
not be quasi-static (extremely slow) so there is no rea- 
son to assume that the gas remains in equilibrium after 
expansion. The entropy of the gas per particle can be 



2.0 



1.6 

o 1.4 

c 
LiJ 



1.2 



1.0 - 




non-equilibrium entropy 
equilibrium entropy 



1.0 



1.2 



1.4 1.6 

box size L 



2.0 



2.2 



FIG. 2: The calculated equilibrium (continuous) and nonequi- 
librium (broken) entropies per particle for an ideal gas in a box 
as a function of the expansion box length L. The nonequi- 
librium state is the result of a sudden expansion from the 
initial state corresponding to L = 1 and To = 4. The energy 
of the gas remains constant in the sudden expansion. As ex- 
pected, the nonequilibrium entropy lies below the equilibrium 
entropy. In time, the former will increase to the latter as the 
gas equilibrates. 



obtained by calculating s{t,L) = —J2kPkit)^^Pkit) for 
the particle under consideration. Henceforth, we will call 
s{t) the entropy of the particle, which shares the prop- 
erty that the irreversible entropy change dis{t) will never 
be negative. All the above discussion about the chemical 
reaction is easily translated to the study of a particle in 
box without any change. The change dip„(i) is caused 
by the transitions between different eigenstates. 

We consider the gas in equilibrium at some initial tem- 
perature Toin in a box of length Lin, which we take to be 
Lin = 1- This is obtained by keeping the box in a medium 
of temperature Toin. The corresponding microstate prob- 
abilities follow the Boltzmann law (/3in = l/Loin): 

Pfe,oq(An,iin) = exp(-/3i„£fe (Li„))/Zo(An, .^in) , 

where Zo(An,-^in) = Efe cxp(-Anefe(iin)) denotes the 
equilibrium partition function. The energy per parti- 
cle in this gas is denoted by ein obtained by replacing 
PkiP,L) by pfe,eq(/3o,-^in) in Eq. ^^■, the corresponding 
pressure is Poin = 2ein/Lin. The equilibrium entropy can 
be obtained by using Pfc,cq(/3o, -^in) in S given in Eq. (|20p . 
The initial temperature Toin for Lin = f is taken to be 
Toin = 4 so that the initial energy £;„ ~ 2.786. 

We now consider equilibrium states having the same 
initial thermodynamic energy ein for different values of L, 
even though the eigenstate energies and their Boltzmann 
probabilities vary with L. The corresponding equilibrium 
entropy Scq(t) as a function of L is shown in Fig. [Hby 
the continuous curve. Since the energy is constant, the 



product PoeqiL)L = 2ein is also constant; see Eq. (l3Tt . 
Thus, Poeq(i) is a decreasing function of L. From the 
slope of the upper curve in Fig. [2] which decreases with 
L, we also conclude that Pacq{L) /Tocq{L) is a decreasing 
function of L. Thus, Tocq{L)L is an increasing function of 
L. However, our calculation to be presented elsewhere^ 
shows that Tocq(i) is also an increasing function of L. 
The eigenstates for a box of size L are given in Eq. (I^Hl) . 

We now consider nonequilibrium states. For this, we 
isolate the box from its medium and consider its free 
expansion as it expands suddenly from Lin to a new size 
L > ijn. Because of its isolation, its energy remains 
£in during this expansion. As the expansion is sudden, 
the initial eigenfunctions il)iin{x) for Lin have no time to 
change, but are no longer the eigenfunctions of the new 
size L; the latter are given by ^pk{x) in Eq. (l29l) for L. 
However, ■!/'iin(a^) can be expanded in terms of tpk{x) as 
a sum over k. We call this the quantum superposition 
principle. The corresponding expansion coefficients bki 
are easily seen to be^^ 



6..(^,An)- ^(^,_^,^,^ sm(-). 



Using bki and pi{/3in, L, Lin), we can determine the prob- 
ability pfc(/3in, L, Lin) for the fcth microstate in the new 
box, which allows us to determine all thermodynamic av- 
erages for the new box. We have checked that the new 
probabilities add to 1 and that the (average) energy after 
the free expansion is equal to Ein to within our computa- 
tional accuracy. Thus, Ae = in the sudden expansion. 
This is consistent with the fact that the gas does no ex- 
ternal work and that no external heat is exchanged. 

Despite this, the free expansion is spontaneous once 
the confining walls have moved. Therefore, the (thermo- 
dynamic) entropy of the gas must increase in this process 
in accordance with the second law. We use pk{Pim L, Lin) 
to evaluate the nonequilibrium statistical entropy, which 
is shown by the dashed curve in Fig. [21 The significance 
of this curve is as follows: Choose a particular value L in 
this graph. Then, the nonequilibrium entropy for this L 
is given by numerically evaluating the sum 

s(/3i„,L,Lin) = ~Y.k Pk{l3in, L, Lin)\npkil3in, L, Lin)- 

This is the entropy after the sudden expansion from the 
initial state at Lin — 1 and follows from the quantum 
superposition principle. Evidently, this entropy is higher 
that the initial equilibrium entropy Soq(/3in,^in)- It is 
also obvious that this entropy has a memory of the ini- 
tial state at Lin = 1 and Loin = 4. Therefore, it does not 
represent the equilibrium entropy. If we now wait at the 
new value of L, the isolated gas in the new box will relax 
to approach its equilibrium state in which its nonequi- 
librium entropy will gradually increase until it becomes 
equal to its value on the upper curve. 



V. 1-D TONKS GAS: A SIMPLE CONTINUUM 
MODEL 

A careful reader would have realized by this time that 
the proposed entropy form in Eq. (fH)) is not at all the 
same as the standard classical formulation of entropy, 
such as for the ideal gas, which can be negative at low 
temperatures or at high pressures. The issue has been 
discussed elsewhere^ but with a very different perspec- 
tive. Here, we visit the same issue from a very different 
perspective that allows us to investigate if and how the 
entropy in continuum models is related to the proposed 
entropy in this work. For this, we turn to a very simple 
continuum model in classical statistical mechanics: the 
Tonks gas,— >^ which is an athermal model and contains 
the ideal gas as a limiting case when the rod length / 
vanishes. We will simplify the discussion by considering 
the Tonks gas in one dimension. The gas consists of r 
impenetrable rods, each of length I lying along a line of 
length L. We will assume r to be fixed, but allow I and L 
to change with the state of the system, such as its pres- 
sure.. The configurational entropy per rod determined 
by the configurational partition function is found to be^ 



Heiv~l)], 



(33) 



where v is the "volume" available per rod L/r. Even 
though the above result is derived for an equilibrium 
Tonks gas, it is easy to see that the same result also 
applies for the gas in internal equilibrium. The only 
difference is that the parameters in the model are also 
functions of internal variables now. 

The entropy vanishes when v = I + l/e and becomes 
negative for all v < I. Indeed, it diverges to — cx) in the 
incompressible limit v = I. This is contrary to the Boltz- 
mann approach in which the entropy is determined by 
the number of microstates (cf. Eq. ([T])) or the Gibbs 
approach (cf. Eq. p4p ) and can never be negative. Can 
we reconcile the contradiction between the continuum en- 
tropy and the current statistical formulation? 

We now demonstrate that the above entropy for the 
Tonks gas is derivable from the current statistical ap- 
proach under some approximation, to be noted below, 
by first considering a lattice model for the Tonks gas 
and then taking its continuum limit. It is in the lattice 
model can we determine the number of microstates. In 
a continuum, this number is always unbounded (see be- 
low also). For this we consider a 1-d lattice Af with iVf 
sites; the lattice spacing, the distance between two con- 
secutive sites, is given by 6. We take iVf >> 1 so that 
Lf = {Nf — 1)5 w NfS is the length of the the lattice 
Af. We randomly select r sites out of Nf. The number 
of ways, which then represents the number of configura- 
tional microstates, is given by 



Wc =N[\/r\{N{-r)\. 



(34) 



After the choice is made, we replace each selected site by 
A -|- 1 consecutive sites, each site representing an atoms 



in a rod, to give rise to a rod of length I = X6. It is clear 
that S also changes with the state of the system. The 
number of sites in the resulting lattice A is 

N ^ Nf + r\ 

so that the length of A is given hy L — {N — l)S « NS 
since iV >> 1. We introduce the number densities (pf = 
r/Nf, Pf = r/N[5 « r/L{ and p ~ r/N5 « r/L. A simple 
calculation shows that S' = In Wc is given by 

S — —N{[p{Ship[S + ln(l — pfS) — pfShi{l — PfS)]. 

This result can also be obtained by taking the ather- 
mal entropy for a polydisperse polymer solution a Bethe 
lattic^^ by setting the coordination number q to be 
q — 2. We now take the continuum limit (5—^0 for fixed 
Pf and p, that is fixed L{ and pL, respectively. In this 
limit, ln(l — pfS) ~ —piS, and p{6ln{l — pfS) ~ —{pfS)'^. 
Use of these limits in S yields 



-r ln(e/pf(5) -^ cx). 



(35) 



The continuum limit of the entropy from the Boltzmann 
approach has resulted in a diverging entropy regardless 
of the value of pf,— a well known result. By introducing 
an arbitrary constant a with the dimension of length, we 
can rewrite S as 



S/r = — ln(e/pfa) + ln((5/a). 



(36) 



in which the first term remains finite in the continuum 
limit, and the second term contains the divergence. The 
diverging part, although explicitly independent of pf , still 
depends on the state of the gas through 6, and cannot be 
treated as a constant unless we assume 5 to be indepen- 
dent of the state of the gas. It is a common practice to 
approximate the lattice spacing 5 as a constant. In that 
case, the diverging term represents a constant that can 
be subtracted from S/r. Recognizing that 1/pf = v — I, 
we see that the first term in Eq. (l36l) is nothing but the 
entropy of the Tonks gas in Eq. ([33l) for the arbitrary 
constant a = 1. However, this equivalence only occurs in 
the state independent constant-(5 approximation. 

As the second term above has been discarded, the con- 
tinuum entropy s^ also has no simple relationship with 
the number (> 1) of microstates in the continuum limit, 
which means that the continuum entropy cannot be iden- 
tified as the Boltzmann entropy in Eq. (fT7|). To see this 
more clearly, let us focus on the centers of mass of each 
rod, which represent one of the r sites that were selected 
in Af. Each of the k sites Xk, k — 1, 2, • • • , r, is free to 
move over if. The adimensional volume jEfl, also called 
the probability and denoted by Z by Boltzmann, Wj^S Qf 
the corresponding phase space Ff is Lf/a"^. However, con- 
trary to the conventional wisdom, ^'^ In jEfl does not yield 
Sc- The correct expression is given by the Gibbs-modified 
adimensional volume jEfl /r!, i.e. 



r\a' 



The presence of r! is required to restrict the volume due 
to indistinguishability of the rods a la Gibbs. For large 
r, this quantity correctly gives the entropy Sc- However, 
this quantity is not only not an integer, it also cannot be 
always larger than or equal to unity, as noted above. 



VI. SEMI-CLASSICAL APPROXIMATION 
PHASE SPACE FOR S 



IN 



The analog of a quantum microstate in classical statis- 
tical mechanics is normally obtained by recognizing that 
in the adiabatic approximation, each small phase space 
cell of volume element dxc (= dpdq in terms of generalized 
coordinates q and momenta p) of size /i" corresponds to 
a microstate,'''^ where s is the degrees of freedom of the 
system. The latter follows from the Bohr-Sommerfeld 
quantization rule for a periodic motion. The adiabatic 
approximation requires the parameters characterizing the 
system to vary extremely slowly. We will assume one such 
parameter A (such as the volume V) so thali^ rA << A, 
where t is the period of oscillation for constant A; the 
system would be isolated in the latter case. In the above 
approximation, the energy of the system will vary very 
slowly and can be taken to be constant over a period of 
oscillation. The action taken over the closed path for the 
constant value of A and the energy is quantized^: 

/ = §pdq= {n+l/2)h. 

This observation is the justification of the above cell size 
of a classical microstate. Thus, the number of "classical" 
microstates is given by 

W = AF//l^ 

where AF is the phase space volume corresponding to 
the system. This allows us to divide the phase space into 
W cells, index hy k, of volume dxk — dxc and "centered" 
at Xk at some time which we call the initial time t — Q. 
We will denote the evolution of the cell at time t by 
the location of its " center" Xk (t) and its volume element 
dxk{t). In terms of the distribution function f{x,t) in 
the phase space, the kth microstate probability therefore 
is given by 



Evidently, 



Pkit) = f{xk,t)dxk{t). 



T,kfi^k,t)dxk{t) = 1 



(37) 



(38) 



at all times. The entropy and the average of any observ- 
able 0{x, t) of the system are given by 

S{t) = -Y.kfi^k,t)dxk{t)\n[f{xk,t)dxu(t)l (39a) 
0{t) = Y.kO{xk,t)J{xk,t)dxk{t), (39b) 

the sum being over all W microstates. While it is easy 
to see that continuum analogs for Eqs. ([55)) and (I39bp 



10 



are easily identified, this is not so for tfie entropy in Eq. 
([39a|).-^ However, it should be obvious that Sj in Eq. ^ 
cannot be a candidate for the statistical entropy S{t). 

It is well known'' that the system in the adiabatic limit 
remains in the same quantum state. For Hamiltonian dy- 
namics, the conservation of the phase space cell volume 
under evolution ensures dxk — dxc for each cell so that 
X = 0. This results in the incompressibility of the phase 
space. In equilibrium, f{xk,t) is not only uniform but 
also constant, and we conclude from Eq. (|38|) that this 
value is / = l/AF. Accordingly, pk — 1/W in equilib- 
rium as expected and we obtain the equilibrium entropy 
S,^ = \nW. 

The situation is far from clear for nonequilibrium 
states. As the example of expansion of the box shows, 
the system is no longer restricted to be in the same 
microstate, which means that the microstate energy is 
no longer a constant and the phase space trajectory is 
no longer closed. Thus, the suitability of the Bohr- 
Sommerfeld quantization is questionable, and care must 
be exercised to identify the microstates. We will adopt 
the following prescription. We consider some equilibrium 
state (uniform f{x)) of the isolated system to identify 
the cell volume dxc = h'^- Once the identification has 
been done, we will no longer worry about its relationship 
with /i*, and only deal with the cells. We then follow 
the evolution of each cell in time in a nonequilibrium 
process during which x = may not hold. Thus the 
volume of each cell may no longer be constant. The pro- 
cess may also result in changes in f{xk,t)M- Indeed, it 
is quite possible that / diverges at the same time that 
dxc vanishes^S However, their product, which determines 
the microstate probability, must remain strictly bounded 
and < 1. In particular, as the cell volume shrinks to zero, 
/ must diverge to keep the product bounded. Thus, the 
divergence^i^ of / alone does not imply that the entropy 
diverges to negative infinity. 

This is easily seen by the following 1-d damped oscil- 
lator, the standard prototype of a dissipative systems^ 
X + 2kx + ujqX = with a (positive) damping coefficient, 
which is chosen such that k > ujq just for simplicity. We 
have the case of aperiodic damping. We will only con- 
sider the long tim e behavi or. It is easy to see that in this 
limit {k' = K + -^/k^ — <^o) 

dXc ~ exp(— 2K't), / ^ exp(-f-2K't), 

and their product remains bounded, as expected. 



VII. JAYNES REVISITED 

Boltzmanni^ provides the following alternative expres- 
sion of the entropyi^i^ in terms of a single particle prob- 
ability p\ for the particle to be in the ith state: 



j(i) _ 



n(l)- 



„(1) 



when all states are equally probable. In this case, 

4' max = ^^ In ^ 

where w is the number of possible states of a single par- 
ticle in the gas. In general, particles are not indepen- 
dent due to interactions and number of possible states 
W < w^ . Accordingly, maximum Gibbs entropy S max 
per particle is less than the corresponding equiprobable 



Boltzmann entropy S 



(1) 



However, Jaynes** gives a 



S^'=-Nj:.l^"]nj^'>, (40) 

not to be confused with that in Eq. ([IJ. Boltzmann is 
only interested in the maximum entropy, which occurs 



much stronger results, see his Eq. (5): 

The equality occurs only if there are no interactions be- 
tween the particles, as we have asserted above. 



VIII. SUMMARY AND DISCUSSION 

Recognizing that there does not exists a first princi- 
ples statistical formulation of nonequilibrium thermody- 
namic entropy for an isolated system in terms of mi- 
crostate probabilities, we have attempted to fill in the 
gap. We use a formal approach (frequentist interpre- 
tation of probability) by extending the equilibrium en- 
semble of Gibbs to a nonequilibrium ensemble, which is 
nothing but a large number A/q of samples of the ther- 
modynamic system under consideration. Accordingly, we 
refer to the ensemble as a sample space. The formal ap- 
proach enables us to evaluate the combinatorics for a 
given set of microstate probabilities. The resulting sta- 
tistical entropy is independent of the number of samples 
and depends only on the probabilities as is seen from 
Eqs. p^ and (^5)1 . Thus, the use of a large number 
of samples is merely a formality and is not required in 
practice. We have shown that in equilibrium, the sta- 
tistical entropy is the same as the equilibrium thermo- 
dynamic entropy: iSo(Xo) = 5*0 (Xq). But we have also 
shown that the statistical entropy is equal to the nonequi- 
librium thermodynamic entropy, provided the latter is 
a state function of the nonequilibrium state variables 
Zo(i): <So(Zo(t)) = So{Zo{t)). We cannot make any 
comment about the relationship between 5o(Zo(f),t) and 
So{Zo{t),t) for the simple reason that there is no way to 
measure or calculate a non-state function S'o(Zo(i),i). 
We should remark here that the standard approach to 
calculate nonequilibrium entropy is to use the classical 
nonequilibrium thermodynamics^! or its variant, which 
treats the entropy at the local level as a state function. 

Some readers may think that our statistical formula- 
tion is no different than that used in the information 
theory. We disagree. For one, there is no concept of in- 
ternal variables ^g in the latter theory. Because of this, 
our approach allows us to consider three levels of de- 
scription so that we can consider three different entropies 
S{Zo{t),t),So{Zo{t)) and 5o(Xo) satisfying the inequal- 
ities in Eq. (|18l) . The information theory can only deal 
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with two levels of entropies. There is also no possibility 
of a residual entropy in the latter. 

For an isolated system in internal equilibrium (pa — p 
for VttIq), just a single sample will suffice to determine 
the entropy as samples are unbiased. The entropy in this 
case is no different than the " entropy" — In p of a single 
sample i^^'^^ 

S{t) ^ (-plnp)Wo = -lnp = lnWo, 

where Wq represents Wo{Zo{t)) or Wo(Ko). However, 
this simplicity is lost as soon as the system is not in 
internal equilibrium. Here, one must consider averaging 
over all microstates. 

Changes in microstate probabilities result in changes in 
the entropy. There are two ways probabilities can change 
within an isolated system, both of them being irreversible 
in nature. One cause of changes is due to the quantum 
nature as seen in the sudden expansion of the box. Here, 
the parameter A (= L) changes non-adiabatically and 
creates irreversibility. The resulting irreversible change 



in the entropy for the 1-d gas has been calculated and 
shown by the lower curve in Fig. [51 The other cause of 
probability changes is due to the " chemical reaction" go- 
ing on among the microstates that brings about equilibra- 
tion in the system. The corresponding irreversible rise in 
the entropy for the gas is shown by the difference between 
the two curves in Fig. [2j The interaction of a body with 
its medium can also result in the changes in microstate 
probabilities, and has been considered elsewhere<^ 



We consider the continuum analog of the statistical 
formulation of entropy and show that the standard for- 
mulation, Sf in Eq. (|31), is not a good candidate of the 
nonequilibrium entropy. It is then argued that the diver- 
gence of / in some cases, see the discussion above, makes 
Sf diverge to — oo, even though the statistical entropy 
remains finite and positive. Thus, Sf cannot be equated 
with our statistical formulation, a generalization of the 
Gibbs formulation. We suggest that our statistical Gibbs 
formulation can be applied to any nonequilibrium state. 
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